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A METHOD AND APPARATUS FOR RE- SIZING AND ZOOMING IMAGES BY 
OPERATING DIRECTLY ON THEIR DIGITAL TRANSFORMS 



TECHNICAL FIELD OF THE INVENTION 
This invention relates to video signal processing and more particularly to a method and 
system for re-sizing images and zooming into and out of images in a way that makes use of 
non-spatial interpretations of the images. 
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BACKGROUND OF THE INVENTION 
Images can be imported into digital systems using opto-electronic transducing devices, 
such as video cameras and computer scanning devices. Also, images can be created within 
digital systems using a variety of computer software programs, such as drawing and animation 
programs. Images are commonly represented and stored in digital systems as arrays of digital 
numbers. 

Images are typically displayed on a suitable display device, such as the cathode ray 
tube ("CRT") of a television or computer monitor, as a two-dimensional spatial representation 
on the surface of the display device. A series of slightly different images can be displayed in 
rapid temporal sequence in order to create the perception of smooth motion as, for example, 
in the case of television image sequences. 

Pixels are relatively small localized fixed regions on the surface of the image display 
device. Displayed images are often composed of many thousands of pixels, wherein each pixel 
has attributes of size, shape, intensity and color. Objects within displayed images are 
represented by groups of pixels. Generally, the pixels are sufficiently close to each other on 
the surface of the display device to ensure that when the displayed image is viewed from a 
sufficient distance objects are perceived to have the same characteristics of shape, texture, 
edges and shading as similar objects in the real world. 

Pixels are often, but not always, arranged on the surface of the display in the form of a 
rectangular grid consisting of uniform rows and columns of pixels. The horizontal and vertical 
spatial resolutions of an image are determined by the average number of pixels per unit of 
distance in the horizontal direction along the rows of pixels and in the vertical direction along 
the columns of pixels. The spatial resolution determines the minimum distance at which an 
image must be viewed so that the human vision system ("HVS") perceives objects within the 
image and not the individual pixels. 

The spatial resolution of a displayed image can be increased by increasing the 
horizontal resolution and the vertical resolution. A high resolution image will yield a more 
natural looking approximation of an original real world image than a low resolution image. 
The quality of the image results from the HVS's perception of the lines and edges within 
displayed images. In low resolution images, the lines and edges are often perceived to be 
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jagged because of the staircase effect caused by rectangular- shaped pixels in the displayed 
image. 

The perception of jagged edges and lines, due to the staircase effect, is less discernible 
in high resolution images because of the reduced size of the rectangular-shaped pixels in the_ 
5 displayed image. 

The finite resolution of the displayed image is a major factor limiting the faithfulness by 
which objects may be represented. 

A digital image is a set of numbers where each number corresponds to a pixel of the 
displayed image. For example, a displayed image might consist of 512 by 640 pixels where 

10 each pixel is characterized by a range of possible luminous intensities and colors. If we 

decompose the color of each pixel into its primary colors of red ("R"), blue ("B") and green 
("G"), then the displayed image may be numerically represented as the combination of the R, 
B and G component images. Each of the R, B and G component images is a monochromatic 
image in which each pixel is characterized only by a digital number representing the luminous 

15 intensity of the pixel A displayed image, consisting of a rectangular array of pixels, may 

therefore be represented as three monochromatic component images, each of which may be 
represented in digital form as a corresponding rectangular array of numbers. We refer to such 
a rectangular array of numbers as a digital image where it is understood that it may represent a 
monochromatic component of a displayed color image or some suitable combination of the 

20 monochromatic components. 

An image display device has a fixed viewing area which is usually a rectangular 
viewing screen. Images displayed on the viewing screen often completely fill the viewing 
screen. Re-sizing of the image means that the image is re-displayed on the viewing screen in 
such a way that the image is perceived by the HVS to have been horizontally or vertically 

25 stretched or compressed. The ratio of the horizontal size of the image after re-sizing to the 

horizontal size before re-sizing is the horizontal scaling factor. Similarly, the vertical scaling 
factor is the ratio of the vertical size of the image after re-sizing to the vertical size before re- 
sizing. Scaling factors greater than unity represent stretching and scaling factors less than 
unity represent shrinking of a displayed image. 

30 The aspect ratio of a displayed image is the ratio of its width to its height. If the image 

is resized using equal horizontal and vertical scaling factors, then the resized image will have 
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the same aspect ratio as the original. By using equal horizontal and vertical scaling factors, 
objects will grow or shrink without changing their shapes and the re-sizing operation will be 
perceived by the HVS as having caused all objects within the image to have moved closer to 
or farther away from the viewer. 
5 Resized images may exceed the dimensions of the viewing screen. Thus, some parts of 

the resized image may lie outside of the viewing screen and may not be visible. A cropped 
image is that portion of the image that is displayed in the viewing screen. 

Zooming is the re-sizing of a set of digitized images or displayed images to create the 
perception in the HVS that objects within the displayed image are growing or shrinking with 
10 time. Zooming creates the illusion that the distances between the viewer and the objects in the 

displayed image are growing or shrinking. 

Smooth zooming is zooming such that the scaling factors increase or decrease 
sufficiently slowly over the set of displayed images to create the perception in the HVS that 
objects within the displayed image are growing or shrinking continuously with time during the 
15 process of zooming. 

If the scaling factors are increasing with time the zooming process is in the zoom-in 
mode and if the scaling factors are decreasing with time the zooming process is in the zoom- 
out mode. For example, a smooth zoom in the zoom-in mode can be achieved in 1 1 
successive frames having scale factors, relative to the first frame, of 1.0, 1. 1, 1.2 . . . 1.8, 1.9 
20 and 2.0, resulting in the final scaling factor of 2. 

Prior art methods for re-sizing and zooming digitized images operate directly on the 
numbers which represent each of the pixels in the displayed image. That is, they operate in the 
spatial domain. 

One prior art method of re-sizing images is the pixel replication method which uses 
25 integer scale factors. The pixel replication method simply copies each pixel some integer 

number of times in both the horizontal and vertical directions. For example, with a scaling 
factor of 3, each pixel of the original image is replicated to form a corresponding 3x3 square 
of pixels in the resized image. A primary disadvantage of pixel replication methods is that the 
jagged staircase effects of the pixelization increase in proportion to the scaling factor. Thus, 
30 pixel replication introduces significant and often unacceptable distortion of the image after re- 

sizing. A further disadvantage of the pixel replication method is that it is limited to enlarging 
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images by integer scaling factors. Therefore, it cannot be used to reduce the size of the image, 
nor can it be used to re-size an image by non-integer scale factors, such as 1.1, 1.2, etc., as 
required for smooth zooming operations. 

Another prior art method is the pixel sub-sampling method, which is used to reducejjie 
size of images using scaling factors that are the reciprocals of integers, such as V 2 , V 3 or V 4 . 
This method implements the scaling factor 1/Q in the horizontal or vertical directions, where 
Q is an integer, by retaining only one of every Q numbers, or pixels, in the digital image in the 
respective directions. The pixel sub-sampling method creates undesirable distortions in 
displayed images. Such distortions lead to the loss of edge definition in objects and are 
noticeable by the HVS in displayed images that are resized using this method, especially for 
large values of Q. 

Abrupt spatial variations of the intensity of the resized image are especially susceptible 
to distortion and are therefore objectionable to the HVS. This distortion can be alleviated if 
the intensity variation between pixels is made to vary smoothly in all directions after pixel 
replication or before pixel sub-sampling. The prior art includes a variety of methods for 
achieving the required spatial domain smoothing. These prior art spatial domain methods have 
a number of disadvantages, including the need for operating directly on each of the pixels in 
the unsealed spatial domain version of the digital image. Other versions of the image, such as 
encoded or compressed versions, in which the image is not spatially represented pixel by pixel, 
cannot be used with the prior art spatial domain smoothing methods. 

In the case of pixel replication, the required smoothed image is often obtained by 
means of a two-dimensional spatial domain low-pass filtering operation that effectively 
smooths out the spatial intensity variations in the digitized image. Two-dimensional spatial 
domain low-pass filtering methods vary in sophistication depending upon the level of 
distortion that can be tolerated in the displayed image. Simple spatial domain weighted 
averaging of regional pixels can be used to smooth the intensity variations in the pixel- 
replicated version of the image, thereby reducing the jagged edge effects. More complicated 
spatial domain filtering methods smooth each number in each row and column of the resized 
digital image. This may require hundreds of multiplication and addition operations to be 
performed on each pixel, which may lead to millions of operations for each resized image. For 
example, if an image of size 512 x 512 is resized to 4096 x 4096 (scaling factor of 8) by pixel 
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replication and a smoothing operation requiring 100 multiplication operations per pixel is 
employed on the resized image, then the total number of multiplication operations that is 
required for smoothing each resized image is 4096 x 4096 x 100. This results in over 
1,600,000,000 multiplication operations per resized image. If the image is cropped to a 512jx 
512 size, then the minimum number of multiplication operations is reduced to512x512x 
100, which is over 25,000,000 per cropped image. Accordingly, the prior art methods for the 
spatial domain reduction of distortion due to pixel replication involves the use of extensive 
hardware and software resources that operate on a pixel by pixel basis. 

In the case of pixel sub-sampling, a two-dimensional low-pass filtering operation is 
performed on the image prior to sub-sampling to smooth out the intensity variations. This 
spatial domain smoothing method has the effect of pre-distorting the image to minimize the 
distortion effects in the resized image following pixel sub-sampling. The effect of this 
smoothing operation is to blur many objects in the image so that their sharp edges are no 
longer distinct after pixel sub-sampling. The required spatial domain filtering operations, like 
those used in pixel replication, vary in sophistication and complexity. When simple weighted 
averaging is used to achieve this filtering, it is often necessary to use hundreds of 
multiplication and addition operations per pixel. For example, to re-size a512x512 image to 
a 64 x 64 image (scaling factor of 1/8) by pixel sub-sampling using a smoothing operation that 
requires 100 multiplication operations per pixel of the resized image, the total number of 
multiplication operations required for smoothing each resized image is 64 x 64 x 100. This 
results in over 360,000 multiplication operations per resized image of size 64 x 64.. Pre- 
distorting the original image using this smoothing operation requires 512x512x1 00 
multiplication operations on the original digital image. This method, like the pixel replication 
method, also requires extensive hardware and software resources that operate on a pixel by 
pixel basis. 

There are prior art methods for re-sizing images in the spatial domain by fractional 
scaling factors, such as 22/73 or 101/100. This capability is required for smooth zooming and 
other re-sizing effects. Almost all of the prior art methods involve a combination of the pixel 
replication and pixel sub-sampling methods described above, along with the appropriate 
smoothing operations. The prior art methods for fractional scaling factors essentially combine 
pixel replication and pixel sub-sampling in a single system, which are referred to in the digital 
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signal processing literature as multirate digital systems. Such methods operate on the image in 
the spatial domain, pixel by pixel, and they employ complicated, computationally intensive 
digital filtering techniques to smooth out the jagged edges and other distortions in the resized 
image. 

Panning is the cropping of a set of digitized images or displayed images to create the 
perception in the HVS that all objects within the displayed image are uniformly translated 
between any two adjacent images in the displayed sequence of images. 

Smooth panning is panning such that displacements of objects in successive displayed 
images are sufficiently small to create the perception in the HVS that all objects within the 
displayed image are moving continuously with time in the displayed sequence of images. 

When objects in displayed images are moving during the zooming or panning 
processes, the perceptions of zooming and panning are retained by the HVS provided that the 
movements over time of objects in images are sufficiently slow. 

Image compression is the process by which digital images are represented by a fewer 
number of bits of information. Image compression allows images to be stored in fewer bits of 
digital memory and thereby reduces the size of computer files that are required for storing 
images. Image compression also allows the compressed version of digital images to be 
transmitted over communication channels at a faster rate than uncompressed images. 

Compressed versions of spatial domain images must be decompressed prior to display. 

Most image compression methods employ mathematical techniques that transform the 
spatial domain representation of the image to a corresponding transform domain version of the 
image that is more suitable for compression. For example, the transform domain version 
typically employs data that has the mathematical property that it is significantly less correlated 
than the data in the spatial domain which is a desirable property for subsequent compression. 
A wide variety of compression methods, including quantization and coding, are routinely 
employed to compress the transform domain version of the image. 

The decompression process attempts to recover the original transform domain version 
in which case the compression-decompression process is said to be lossless. In practice the 
decompression algorithms often recovers only an approximation to the original spatial domain 
image, and the process is said to be lossy. 
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The transform domain versions of images do not necessarily contain data that 
corresponds to spatial domain pixels and therefore may not be operated upon by the above 
mentioned prior art re-sizing and zooming methods to achieve re-sizing and zooming of 
displayed images. 

International standards have been developed for accomplishing compression of digital 
images. For example, the MPEG 1 and 2 compression standards are the most widely used for 
digital image sequences while JPEG is commonly used for still images. 

These standards use the discrete cosine transform (DCT) to convert spatial domain 
images to the transform domain and the inverse discrete cosine transform (EDCT) is used to 
convert the transform domain version to the corresponding decompressed spatial domain 
image. Prior art methods of re-sizing, zooming or panning displayed images, that have been 
compressed using the MPEG standards, operate on the spatial domain image. 
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SUMMARY OF THE INVENTION 

The present invention provides a method of re-sizing a spatial domain image by 
modifying the transform domain method that is used to transform the transform domain 
version of the image to the spatial domain version of the image. By modifying the transform 
domain method, there is no need to perform further pixel-by-pixel image re-sizing, zooming or 
panning operations on the image in the spatial domain. The detrimental effects caused by re- 
sizing an image in the spatial domain, such as jagged edges, blurring, aliasing or other image 
degradations, are reduced in the present invention. 

The standard transform method, which transforms the image from the transform 
domain to the spatial domain, is modified. Transformation of the transform domain image, 
using the Modified Transform Method, produces an altered spatial domain image. These 
modifications to the transform operation are performed by the Modified Transform Method in 
such a way as to achieve desirable re-sizing or zooming effects in the altered spatial domain 
image after transformation. 

It is a technical advantage of the present invention to provide a method of modifying 
the appearance of the spatial domain representation of a digital image by applying a Modified 
Transform Method to the transform domain version of the image. The re-sized spatial domain 
image can be achieved by modifying a large number of transformation methods, such as the 
Inverse Discrete Cosine Transform method, the Inverse Discrete Fourier Transform method 
and the Inverse Discrete Wavelet Transform method. Furthermore, the invention can be 
applied to the block images (that is, blocks) used in standard image compression techniques, 
such as MPEG or JPEG. 

It is a further technical advantage of the invention that no further modification of the 
spatial domain image is required after performing the Modified Transform Domain Method. 
This precludes the need for extensive pixel-by-pixel modification of the spatial domain image 
in order to re-size or zoom. 

An additional technical advantage of the invention is to provide for modification of 
digital images without using extensive hardware and software resources. Re-sizing and 
zooming modification of the spatial domain representation of an image requires expensive and 
time consuming pixel-by-pixel processing. In the present invention, the spatial domain images 
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created by the Modified Transformation Method are in their re-sized or zoomed form and are 
capable of being displayed without further modification or processing. 

The foregoing has rather broadly outlined the features and technical advantages of the 
present invention in order that the detailed description of the invention that follows may be__ 
better understood. Additional features and advantages of the invention will be described 
hereinafter which form the subject of the claims of the invention. It should be appreciated by 
those skilled in the art that the concept and the specific embodiment disclosed may be readily 
utilized as a basis for modifying or designing other structures for carrying out the same 
purposes of the present invention. It should also be realized by those skilled in the art that 
such equivalent constructions do not depart from the spirit and scope of the invention as set 
forth in the appended claims. 
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BRIEF DESCRIPTION OF THE DRAWINGS 
For a more complete understanding of the present invention, and the advantages 
thereof, reference is now made to the following descriptions taken in conjunction with the 
accompanying drawings, in which: 

FIGURE 1 A is a representation of the transformation of an image from the spatial 
domain to a transform domain and is referred to herein as a <£> - Transform Method; 

FIGURE IB is a representation of the transformation of an image from a transform 
domain to the spatial domain and is referred to herein as a ¥ - Transform Method; 

FIGURE 2 shows the results of performing a <E> - Transform Method, a? - Transform 
Method and a Modified *P - Transform Method; 

FIGURE 3 shows a Modified <P - Transform Method using an augmented transform 
domain image; 

FIGURE 4A shows a resized image which is too large for the display window; 
FIGURE 4B shows the resized image of FIGURE 4 A cropped to fit the display 
window; 

FIGURE 4C shows the resized image of FIGURE 4 A cropped to fit some sub-portion 
of the display window; 

FIGURE 5 A shows the resized image divided into discrete blocks; 

FIGURE 5B shows a display window moving with respect to the blocks of the resized 
image of FIGURE 5A; 

FIGURE 6 illustrates enlarging a digital image using the method of the present 
invention; 

FIGURE 7 illustrates shrinking a digital image using the method of the present 
invention; 

FIGURE 8 shows a television viewing system having viewer control. 
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DESCRIPTION OF THE PREFERRED EMBODIMENTS 
A digital image is an array of digital numbers wherein each digital number in 
the array represents a corresponding pixel of the displayed image. Both the digital image and 
its displayed version are spatial-domain images and may be represented mathematically by a__ 
5 real-valued function ^( w p n 2 ) where n x and n 2 are non-negative integers that point to the 

number in the column and row of the digital image array, as shown in FIGURE 1 A. In 
the displayed image, n x and n 2 point to the corresponding displayed pixel, for a rectangular 
digital image of size columns by N 2 rows, 0<,ji 1 <(N 1 ~\^ and 0zn 2 <(N 2 -l). The spatial- 
domain is referred to as D 10 in Fig 1 A and contains the digital image 14. 
10 We say that n v ti 2 €D. The present invention provides a method of re- sizing a digital 

image x (^v n 2j to create a re-sized version xr^i v n^ of the digital image by modifying a 
Transform Domain Method. By means of the re-sizing method, this invention further provides 
methods for zooming and panning of digital images and thereby of displayed images. The 
Method may be used to achieve re-sizing, panning and zooming without operating directly on 
15 the spatial domain image. 

A set of mathematical operations that transforms a spatial-domain digital image 
xfn v n 2 } to a transform domain version 0[x(« p n 2 | of the image is referred to as a O- 
Transform Method if the transform domain data $[x^ 19 n 2 )] ^ oes not ^ ave a one-to-one 
correspondence with spatial pixels in the spatial domain image xjn^n^. Conversely, a set of 
20 mathematical operations that transforms a transform-domain version of an image to a spatial 

domain version of the image is referred to as a Transform Method. 

The ¥ Transform Method may be applied to the transform domain version of an 

image 

<&[x^ p 77 2 )] to obtain the spatial-domain image YjG^x^, « 2 )j] . 
25 If the application of the O- Transform Method, followed by the application of the 

Transform Method, leads to the original spatial domain digital image x{n v « 2 ), then the *F- 
Transform Method is the Inverse Transform of the O - Transform Method and we say that 

There exists a wide variety of operations on spatial-domain images that transform them 
30 from the spatial-domain to a transform-domain. An important class of <X> -Transform 

Methods yields a transform domain image of the form 
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X(k v k 2 ) }=<E>^c(n v n 2 ^ where ^and ^ 2 are the so-called transform-domain variables in the 
transform-domain Q D . Such transforms may be written in terms of their basis functions 
b (k ly k 2 , n x , /7 2 ), in the form 

x ^i k ^Yl H xlp v n 2 )b(k v k 2 ,n v n 2 ), where * 1 * 2 eQ Z) 

and is summarized in FIGURE 1 A where the transform operation 16 is shown yielding a 

transform-domain representation Xlk^k^ of the digital image x{{i v n^ 

In FIGURE 1 A, X(k v £ 2 )l8 is of size K x xK 2 and the transform-domain Q. D 12 is shown. 

The basis function b ^ , k 2 , n } , n 2 ) is independent of the digital image 
x ftv n 2)> ^ ut ^ oes depend on the type of transform that is selected. For example, the 
Discrete Fourier Transform and the family of Discrete Cosine Transforms have different and 
particular basis functions. Also, the basis functions of the Discrete Fourier Transform and the 
Inverse Fourier Transform are different. 

The transform-domain image X(k v k^j is an image that is not usually viewed or 
displayed because the values of a transform-domain image Xfk^k^ do not correspond to 
pixels in the displayed version of the digital image. 

There exists a corresponding set of ¥ - Transform Methods that operate on 
transform-domain images to transform them from the transform domain to the spatial-domain. 
An important class of ¥ - Transform Methods yields a spatial domain image of the form 

x^? 1 ,/i 2 )= , F pf^tj,^ where n Y and n 2 are the so-called spatial-domain variables in the 
spatial-domain D. Such transforms may be written in terms of their basis functions 
b(k v k 2 ,n v n^j in the form 

x { n i n 2)=12 X) X(k v k 2 \bfk v k 2 ,n v n 2 YwheTe n }7 n 2 e D. 

As summarized in FIGURE IB, the *F - Transform Method *P [A^, * 2 )] 20 is shown yielding 
a spatial-domain representation of the digital image xfn y ,n 2 } 14'. 

If the operations in FIGURES 1 A and IB are applied in sequence and images 14 and 14' are 
identical, then ¥ pf(A p A 2 )] is the inverse transformation of 
O /i 2 )] md therefore HP = <t> -1 . 
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In the prior art, a re-sized displayed image is achieved by operating on the spatial- 
domain digital image, employing methods that operate on each number in the spatial-domain 
array of the digital image 14' in FIGURE IB. As further described above, the result of these 
prior art methods is that a very large number of arithmetic operations are required to avoid _ 
undesirable jagged edges and other image distortions that result from the down-sampling and 
up-sampling operations which are required by spatial domain re-sizing. 

The present re-sizing invention avoids the requirements for spatial-domain operations 
on each number in the spatial-domain array xf?7 v n^) 14' and also avoids the need for the 
associated spatial-domain smoothing operations. The present invention is to modify the *F - 
Transform Method and is therefore referred to as the Modified *F -Transform Method. Upon 
applying the Modified V -Transform Method to the transform-domain image X(k v k^ the 
required re-sized spatial-domain digital image xrfn v njj is created and may be displayed. 

Figure 2 illustrates the operation of the preferred embodiment of the re-sizing method 
of the present invention. Image x(« p " 2 ) 22 represents the spatial-domain digital image prior 
to re-sizing. Images 22, 24 and 34 may be entire displayed images or sub-image regions of the 
displayed image. If the sub-image region is spatially-rectangular, it is an image-block or block. 
The transform-domain image X(k y , k^j 24 is in the transform domain Q. D The unmodified *F - 
Transform Method may be employed as an inverse transform *F = <t> -1 to convert the 
transform-domain image Xfp v kjj 24 back to the original spatial-domain digital image 22. 

By the preferred re-sizing embodiment of this invention, a Modified ¥ - Transform 
Method 36 is applied to the transform domain image Xfk l9 ty 24 to create the re-sized spatial- 
domain digital image xrfe v n^} 34 where the Modified ¥ - Transform Method is written 

and where p Aj] 36 is shown in FIGURE 2. 

The Modified ¥ -Transform Method is any set of modifying operations 
^a/od^^I'^)] on an y ^ ata or algorithms of the unmodified ¥ - Transform Method 
¥ pSr^fcj,^)] that causes the image xrfp l9 n^j 34 to be a predetermined re-sized version of the 
digital image xfn v n^j 22. Such operations include, but are not limited to, changing the size of 
the data array X(k ly k 2 ) 24 by augmenting the array with additional data and also include, but 
are not limited to, modifying the parameters of the transformation, such as its basis functions. 
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The data points of the transform-domain image X(k v k 2 )24 may be complex numbers, 
depending on the selected <& - Transform Method. 

A preferred embodiment of this invention is to modify the ¥ - Transform Method by 
modifying the basis functions bfk v k 2y n v njj to create a modified set of basis functions 
b MOD$v k t* n v n 2\ and > in pa* 1 * 011131 '* such ^ 



k x k 2 



s x s 2 



where b MOD ik iy k 2i n v n^j are the modified basis functions and Sj and s 2 are real numbers that 
are called the horizontal and vertical scaling factors, respectively. The basis functions are 
modified as part of the Modified *F - Transform Method. 
For example, if the basis functions are of the form 



b(k v k 2 ,n v n 2 )= cos 



2N } 



cos 



' (2n 2 + l)k 2 n\ 
, 2N > J 



then the modified basis functions are of the form 



WM2>"l' W 2 ) =COS 



(2",+l) 



n 



2N t 



cos 



(2« 2 + l) 



( k ^ 

S S 2/ 



When the value s x is greater than unity, the above scaling of the basis functions causes 

the re-sized imager^, « 2 )3 4 to be stretched in the horizontal direction relative to xfn l9 n^22. 
When the value of s x is less than unity, the re-sized image xr£i l ,n 2 )34 is shrunk in the 

horizontal direction relative to x^ p « 2 )22. Similarly, s 2 achieves stretching or shrinking 

of xrty v n^ 34 in the vertical direction, depending on whether s 2 is greater or less than unity. 

If s x = s 2 then the stretching or shrinking of xr^t p n 2 ) 34 relative to xjri v 22 occurs 
without altering the aspect ratio of the re-sized image relative to the original and without 
altering the perceived shapes of objects in displayed versions of those images. 
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A further modification to the VP - Transform Method related to re-sizing images, for 
the case when s Y and s 2 are greater than unity, is to increase the size of the transform-domain 
image X(k l9 k^j 24 by generating additional synthetic data points so that the overall size of the 
augmented transform-domain image Xfjc^k^ 24 and the re-sized spatial domain image 
5 xrfn v n^j 34 are predetermined. In Figure 3, the original array of data points CI 1 30A of the 

¥ - Transform Method is augmented by additional generated arrays C12 30B, C21 30C and 
C22 30D of synthetic data points that serve to modify and increase the size of the transform- 
domain image Xfk v k 2 } 24 and create the modified transform-domain image X A[/G (k v k 2 ) 32. 
The synthetic data in the additional generated arrays C12 3 OB, C21 30C, and C22 30D may be 
10 generated by means of operations on the data points in Cll 3 OA or by some other means or 

they may be set to zero value. 

The sizes of rectangular arrays CI 2 30B, C21 30C, and C22 30D may be chosen so 
that the number of rows P x of 32 satisfies 

1 5 and the number of columns P 2 of 32 satisfies 

P 2 > s 2 N 2 

For example, if N x = N 2 = 8 for an image-block and the desired scaling factors are 
s Y -s 2 ~ 1.5, then we choose P 12 > 12 for the size of the image-block xrfn v n 2 ) j34. If s l N 1 is an 
integer, then the number of columns P x of image 32 may be chosen equal to s } N } and if s 2 N 2 
20 is an integer then the number of rows P 2 of image 32 may be chosen equal to s 2 N 2 . The 

transform-domain image 32 is referred to as the Augmented Transform Domain Image and 
denoted X AUG (k v k 2 y 

To obtain the re-sized spatial domain image xr\n v n^j3A y the Augmented Transform 
Domain image X AUG i^ v k^j 32 of transform domain data is transformed using a Modified - 

25 Transform which employs the modified basis functions b MOD (k v k 2 , n v n^. The Modified VP - 

Transform can then be expressed mathematically in the form 
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where k l7 k 2 e£l D , implying that the summation over the transform domain variables k v k 2 is 

over a suitably chosen region of the transform domain Q D . By using the above Modified 

Transform Method to obtain re-sized spatial domain image xrfn v n^ 34, jagged edges 

and other undesirable aliasing effects are avoided and there is no need for extensive smoothing 
operations on the spatial domain image xrty v n^j 34 to reduce jagged edges and other 

distortion effects in the re-sized image. 

For example, a spatial-domain image xfn^n^ 14 of size N x =N 2 = 128 may be re-sized 

by scaling factors s x =s 2 = 2 to obtain a re-sized image xrfp l9 n^j 34 of size P x =P 2 = 256 by 

operating on the transform domain image Xfk^k^ 24. The data array CI 1 30A represents the 

array of X(k v k^j 24 and is of size 128 x 128 for the unmodified Inverse Discrete Cosine 

Transform. The augmented arrays C12 30B, C21 30C and C22 30D may be chosen as zero- 
valued arrays of size 128 x 128 (represented as "0" below) so that the augmented array 





c l2 




X(k v k 2 ) o 


c 2l 


c 




0 0 



which is of size 256 x 256. The Modified T - Transform Method is selected as the Inverse 
Discrete Cosine Transform operation on this Augmented Transform Domain Image 
X AU Jk v k^ using the following modified basis functions with s x =s 2 = 2 



(2« 1 + 1) 



71 



2W X 



COS 



(2k, + 1) 



7T 



2N n 



so that the Modified V - Transform Method is 
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P,=255 /> 2 =255 
Jt,=0 A^=0 



cos 



(2*2 + 1) 



/c 2 



7C 



It is an advantage of this invention that objects in the re-sized image xrfn v n^j 34 do not 

exhibit the distortion effects of pixel replication and the edges of objects are visually smooth. 
This is achieved without spatial-domain smoothing filters. 

The above example can be implemented with small block images. For example, the 
original image jc(w p /j 2 ) could be a block image of size 8x8 and, with s } =s 2 =2, the re-sized 

image block would be of size 16 x 16. The total re- sized and displayed image could be 
composed of many such re-sized image blocks. 

FIGURE 4A illustrates the case when image 42 is re-sized such that the re-sized image 
xrfn v w 2 ) 44 is too large for the display window 46. A similar problem occurs if the image is 

to be used in a desired sub-area of the display window 46. In some cases, the re-sized image 
may fit within the overall display area but may be too large for the desired sub-area of the 
display. In FIGURE 4A, the original image xfn v n^} 42 of size N Y xN 2 fits within the display 

window 46 which is of size W { x W 2 . Display window 46 can represent either the entire area 

of a display or a sub-area of the display. Re-sized image xrfyi v njj 44 is an enlarged version of 

image 42 produced using the Modified V - Transform Method with scaling factors s { and s 2 

as disclosed above. Re-sized image 44 is of size R x xR 2 , where R^ =s x N } and R 2 = s 2 N 2 . 

In this example, re-sized image 44 is larger than display window 46, causing portions of the 
enlarged image to fall outside of display window 46. FIGURE 4B shows cropped image 
xcr(n l7 rii) 48 which comprises a portion of re-sized image 44, such that the cropped image 48 
fits within the display window 46. The cropped image 48 is of size CR^CR^ where 

CR X ~N l and CR 2 -N 2 so that cropped image 48 approximates the dimensions of original 
image 42. As shown in FIGURE 4C, re-sized image 44 also may be cropped so that the 
dimensions, CR{ and CR 2 \ of cropped image 50 are not equivalent to those of original image 
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42. In this case, cropped image 50 can be of any suitable size for viewing on a sub-portion of 
the display, such as display window 86 of FIGURE 8, which shows a zoomed-in version 84* of 
image 84. 

In cases where the resized image is larger than the display window, instead of cropping 
5 the resized image, it may be desirable to pan the resized image so that sequential portions of 

the entire resized image may be viewed in the display window. 

As shown in FIGURE 5 A, resized image 44 can consists of blocks 52 which 
correspond to discrete spatial regions of resized image 44. Blocks 52 are of dimension B l x B 2 
and may be of various sizes, from single pixels to multiple pixel rows and columns. 
10 FIGURE 5B shows display window 46 panning over resized image 44. Display 

window 46 covers a portion of the blocks 52 of the resized image 44. Blocks 52 which are 
within display window 46 can be viewed by a user. The resized image 44 is panned by 
sequentially moving blocks 52 across the display window 46. For pixel-level panning, blocks 
52 correspond to individual pixels of resized image 44. The image is panned by sequentially 
15 adding one row or column of pixels to an edge 54 of display window 46 and removing one 

row or column of pixels from the opposite edge 56 of display window 46. A shift vector 58, 
represented as d pan = ^ horiz d ver ^ pixels/frame, defines the movement of re-sized image 

xr(n v n 2 ) 44 with respect to display window 46, where d horiz is the number of blocks in the 

horizontal dimension and d is the number of pixels in the vertical dimension. 

20 For pixel-level panning, d horgz = -1,0 or +1 and d^ = -1,0 or + 1 , so that the 

display window moves one row and column per frame. Resized image 44 can also be panned 
using multiple pixels. In multiple-pixel-level panning, multiple rows and columns of pixels 
may be added and removed at the edges, so that the components of the shift vector are: 

d horiz=~ m horiz* 0 ° r +m hori Z ^ d veri = ~ m verV 0 ° r +™veri> 

25 where m hori7 and m vert represent multiple columns and rows of pixels. In block-level panning 

m horiz > the horizontal block size, and m vert =M 2 , the vertical block size. 

Panning can be accomplished using the Modified *F - Transform Method disclosed 
above. By identifying groups of blocks in the transform domain image which are within or 
partially within the display window, the Modified ¥ - Transform Method is applied to each of 
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those blocks to produce that portion of the resized image 44. A second group of blocks is then 
identified which corresponds to those blocks which will overlap the display window in the next 
frame after the display window is moved by the appropriate shift vector. The Modified ¥ - 
Transform Method is then applied to the second group of blocks to produce a shifted portion of 
the resized image 44. 

The cropping and panning functions described above will be understood to apply to 
images which have been resized using scale factors that are greater than, less than or equal to 
unity. Additionally, the re-sizing, panning and cropping functions may be applied to images in 
real-time. For example, an image may be received by a digital system via an opto-electronic 
transducing device, such as a video camera, the image may then be compressed using an 
appropriate MPEG standard. The compressed image can then be inverse transformed using a 
Modified *F - Transform Method to achieve a displayed image which has been resized, 
zoomed, panned or cropped according to the present invention. 

FIGURE 6 illustrates the re-sizing method of the present invention for the case in which 
the original spatial domain image x(n v n 2 ) 600 is enlarged by a non-integer scale factor. In this 

example, the scale factor is ^=^=1.5 . The re- sizing shown in FIGURE 6 also includes an 

example of a re-sizing operation that is combined with a cropping operation to produce the final 
resized image xcr(n v n 2 ) 602. Original image 600 is transformed to the transform domain using 

the Discrete Cosine Transform 3>[x^? l ,/? 2 jjj 604 to produce transform domain image Xfk l9 k^j 

606. This transformation is represented mathematically as: 

7 7 
n] =0 n 2 -0 

where the original image size corresponds to N l2 = 8 in this example. 

Transform-domain image 606 is then augmented with synthetic data points DP 608 to 
create augmented transform image X AUG (k v k 2 ) 610. Augmented transform image 610 is of size 

P Y xP 2 , where P x 2 z s l2 N l2 . For a scale factor of 1.5 and an original spatial domain of size 

8x8, P l2 must be greater than or equal to (1 .5) (8) = 12 . In this example, instead of using an 

augmented transform image of size 12 x 12, the augmented transform image 610 is of size 16 x 



( (2« 1+ 1)tcV 


cos 


' {2n 2 + \)nk^ 


I (2)(8) J 




{ (2)(8) j 



WO 00/16263 



21 



PCT/US99/21298 



16. This satisfies the requirement P l2 £ 12 and may be chosen because fast transforms exist for 
this size. A value of P Y 2 2> 12 may also be selected due to consideration of such factors as the 

hardware or software used by the system. 

Augmented transform image 610 is then transformed using a Modified ¥ - Transform 
Method ^mod^aug&M] 612 - Modified ¥ - Transform Method 612 is derived by scaling 
the basis functions b(k v k 2 ,n x ,n 2 ) of the unmodified Transform Method ¥pr(Aj,£ 2 )] by 



scale factors s ia . This creates modified basis function b MOD (k 1 ,k 2 ,n l ,n 2 )=b 



k k \ 
— , — ,n n 



Applying the Modified *F - Transform Method 612 to the Augmented Transform Domain 
Image 610 directly yields the re-sized image xr(n v n 2 ) 614 in the spatial domain. For this 

example, using a modified Inverse Discrete Cosine Transform operation as the Modified ¥ - 
Transform Method, the inverse transformation of the augmented image 610 is represented 
mathematically as: 



15 15 



(2nj + 1X^/1.5) 
(2K8) , 



cos 



' (2w 2 + l)jt(i 2 /1.5)' 



(2)(8) 



Resized image 614 is of size 16 x 16 and includes replicated borders 616 and 618 which 
result from augmented image 610 being larger than size 12 x 12. A 12 x 12 augmented 
transform image would correspond to the scale factor s x 2 =1.5 . However, by choosing an 

augmented image 610 of a larger size, an excess amount of resized image is created following 
the Modified *F - Transform Method. This excess image that is contained in borders 616 and 
618 can be cropped to give cropped resized image 602 of size 12 x 12, corresponding to a 1.5 
scaling of the original 8x8 image 600. 

Similarly, FIGURE 7 illustrates the re-sizing method of the present invention for the 
case in which the original spatial domain image x(n v n 2 ) 700 is reduced by a non-integer scale 

factor to create the final cropped and resized image xcr(n x ,n 2 ) 702. In this example, the scale 

factor is 3 / 4 . Original image 700 is transformed using transform operation 0[x(/2 p /? 2 )J 704 to 
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give transform image X(k v k 2 ) 706. Since this re-sizing is not an enlargement, transform image 

706 does not have to be augmented. Instead, the Modified ¥ - Transform Method 
¥ M0 ^pf(£j,A; 2 )] 708 is applied directly to transform image 706 to give resized image xr(n v n 2 ) 

710. 

Resized image 710 contains excess image in borders 712 and 714 due to the scaling of 
the Modified *P - Transform Method 708. By cropping resized image 710 to remove excess 
border 712 and 714, cropped and resized image xcrjn v n^j 702 is created. Image 702 

corresponds to a 3 A scaling of original image 700. 

FIGURE 8 shows a system 80 in which a viewing device 81, such as a television or 
personal computer, has a picture 82 with a plurality of images 801, 802 and 803. A user (not 
shown) using a remote control 83 or other device, such as a computer mouse, can select a 
particular image for further zooming. For example, portion 84 of image 82 is selected by 
adjusting outline 85. This selected image can then be displayed on the screen in place of image 
82 or, optionally, in a separate window 86. The image displayed in window 86 can then be 
resized or zoomed in or out. As discussed above, the viewer could also elect to pan the image 
of screen 82 (or of screen 86) either row-by-row, column-by-column, block-by-block or in a 
combination of these. 

Although the present invention and its advantages have been described in detail, it 
should be understood that various changes, substitutions and alterations can be made herein 
without departing from the spirit and scope of the invention as defined by the appended claims. 
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WHAT IS CLAIMED IS : 

1 . A method of modifying a transform domain representation of a digital image 
comprising the steps of: 

modifying aT- Transform Method associated with said transform domain to produce 
a Modified *F - Transform Method; and 
5 applying said Modified *F - Transform Method to said transform domain representation 

to create a re-sized version of said digital image. 

2. The method of claim 1, wherein said modification of said - Transform 
Method comprises modifying basis functions of said transform method. 

3. The method of claim 1, wherein said re-sized digital image is a zoomed version 
of said digital image. 

4. The method of claim 3, wherein a plurality of said zoomed version digital images 
are combined to create a smooth zoomed version of said digital image. 

5. The method of claim 1, wherein said re- sized digital image is a panned version of 
said digital image. 

6. The method of claim 1, wherein said modification to said ¥ - Transform 
Method comprises a scaling factor. 

7. The method of claim 1, wherein said *F - Transform Method is based on the 
Inverse Discrete Cosine Transform. 

8. The method of claim 1, wherein said ¥ - Transform Method is based on the 
Inverse Discrete Wavelet Transformation. 
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9. The method of claim 1, wherein said modification to said ¥ - Transform 
Method includes generation of a plurality of synthetic data points. 

10. The method of claim 9, wherein said synthetic data points are zero valued. — 

11. The method of claim 9, wherein said modification of said *F - Transform 
Method comprises modifying basis functions of said transform method by a scaling factor; and 

wherein said plurality of synthetic data points is proportional to said scaling factor. 

12. The method of claim 11, wherein said *F - Transform Method is based on an 
Inverse Discrete Cosine Transformation operation. 

13. The method of claim 11, wherein said *F - Transform Method is based on an 
Inverse Discrete Wavelet Transformation operation. 

14. The method of claim 1, wherein said *P - Transform Method can be expressed 
mathematically as 

where b(k v k v n l9 n^) is a basis function of said transform operation. 

15. The method of claim 1, wherein said transform domain representation and said 
¥ - Transform Method complies with a standard selected from the group consisting of: 

MPEG; 
JPEG; and 
5 MJPEG. 

16. The method of claim 2, wherein said transform domain representation and said 
basis function comply with a standard selected from the group consisting of: 

MPEG; 
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JPEG; and 
5 MJPEG. 

17. The method of claim 1, further comprising the step of: 

cropping said re-sized image to remove an excess image, wherein said excess image 
comprises a portion of said re-sized image. 

18. The method of claim 17, wherein said excess image comprises a duplicated 
portion of said resized image. 

19. The method of claim 1, further comprising the step of: 

cropping said resized image to create a cropped image having special dimensions, 
wherein said special dimensions are selected so that said cropped image fits within a specified 
area of a display. 

20. The method of claim 2, further comprising the step of: 

cropping said re-sized image to remove an excess image, wherein said excess image 
comprises a portion of said re-sized image. 

21. The method of claim 9, wherein said ¥ - Transform Method comprises an 
Inverse Discrete Cosine Transform. 

22. The method of claim 9, wherein said *F - Transform Method comprises an 
Inverse Discrete Wavelet Transform. 

23. The method of claim 9, wherein said transform domain representation and said 
Modified *F - Transform Method comply with a standard selected from the group consisting 

of: 

MPEG; 
5 JPEG; and 

MJPEG. 
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24. The method of claim 9, further comprising the step of: 

cropping said re-sized image to remove an excess image, wherein said excess image 
comprises a portion of said re-sized image. 

25. The method of claim 9, further comprising the step of: 

cropping said re-sized image to create a cropped image having special dimensions, 
wherein said special dimensions are selected so that said cropped image fits within a specified 
area of a display. 

26. A system for modifying a video display, said system comprising: 
means for displaying video data; 

means for accepting an input from a user pertaining to a desired modification to said 
displayed video data; and 
5 means, controlled in part by said accepting means, for modifying a ¥ - Transform 

Method to accomplish said desired modifications. 

27. The system of claim 26, wherein said displayed video data is modified by a scale 

factor. 

28. The system of claim 26, wherein said accepting means comprises: 
means for selecting an area of said displayed video data to be modified. 

29. The system of claim 26, wherein said accepting means comprises: means for 
panning a selected area of said displayed video data. 

30. The system of claim 26, wherein said accepting means comprises: 

means for selecting a particular area of said display means on which to display said 
modified video data. 
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31. The system of claim 26, wherein said accepting means comprises: 
means for selecting an area of said displayed video data to be modified; and 

means for selecting a location on said display means for displaying the modified video 
data. __ 

32. A system for displaying multiple images comprising: 
means for displaying an image; 

means for selecting a portion of said displayed image; and 

means for concurrently displaying, on said display means, the image and the selected 
5 portion, wherein said selected portion is modified using a Modified *F - Transform Method. 

33. The system of claim 32, wherein said means for selecting a portion of the image 
comprises: 

a television remote control. 

34. The system of claim 32, wherein said means for selecting a portion of the image 
comprises: 

a computer pointing device. 

35. The system of claim 32, wherein said modified selected portion has been resized 
using said Modified *P - Transform Method. 

36. The system of claim 32, wherein said displayed selected portion has been panned 
using said Modified ¥ - Transform Method. 

37. The system of claim 32, wherein said displayed selected portion has been 
zoomed-in using said Modified *F - Transform Method. 

38. The system of claim 34, wherein said displayed selected portion has been 
zoomed-out using said Modified ¥ - Transform Method. 
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